Application software designed to implement business systems can be described as having three tiers or elements. The application software generally comprises a database, to store relevant business data, a business logic/transaction management module, which carries out appropriate manipulation of the data contained in the database, and a user interface, through which a user can interact with the software application.
It will be understood that the word “database”, in the present context, denotes any type of persistent store, such as, for example, a relational database, a hierarchical database, an object oriented database, or a file system.
A number of so-called “fourth generation” programming languages have been developed to assist in the creation of business application software. Such fourth generation languages are characterised, in part, by the ability to generate appropriate database layouts, transaction handling routines and user interfaces when provided with an abstract input structure. That is, a fourth generation programming language allows a user to describe their business logic (business rules) in general abstract terms, and from these business rules, all appropriate software code, database structures and user interfaces are generated by the language. In other words, the database layout, software code and user interface is derived from a high level and abstract description. The user does not specify the final structure explicitly.
In the present context, the term “user” refers to a software architect or developer, and/or a system administrator of the aforementioned business application software. A “user” in this context does not refer to end-users of the final software application, such as bank tellers performing transactions from their terminals.
A user of the business application software perceives a fourth generation language as a set of abstract entities, not as a set of database and business logic implementation artifacts. This high-level view relieves the user from the need to address irrelevant (from the user's point of view) details such as, for example, the database table layout and properties, the details of a given database administration, etc.
One example of a fourth generation programming language is the Enterprise Application Environment (EAE), which is developed by Unisys (a corporation with headquarters in Blue Bell, Pa.). In EAE, the user (i.e. the software developer) deals with concepts such as “Ispecs” and “Events”
Ispecs and Events are abstract entities in EAE. “Ispec” is shorthand for “Interface Specification”. An user's system is made up of one or more Ispecs. Each Ispec can be either an “Input” Ispec, or an “Output” Ispec, or an “Input/Output” Ispec. An Input Ispec contains a user interface, but no persistent data. An Output Ispec contains persistent data, but no user interface. An Input/Output Ispec contains both a user interface and persistent data. A user creates an application system by creating Ispecs to perform various functions.
For example, if the application requires a menu (for example, a banking application may have a menu such as, “View Account Details; Transfer Funds; Pay Bills”) the menu has no persistent data associated with it, so it will be represented as an Input Ispec. On the other hand, a person's bank account will have persistent data, such as the current account balance, and the history of account transactions, but will not have a user interface.
Thus, a bank account would be better represented as an Output Ispec. The same banking application may have a form that requires input from an end user to transfer money between accounts. The form is a user interface, but the data entered (such as the account from which to transfer the money, the account to which to transfer the money, and the amount of money to transfer) needs to be stored persistently, so the form would best be represented as an Input/Output Ispec.
Thus, Ispecs are abstract entities with certain well-defined properties and behaviours in terms of which a user can design an application, which abstracts away the low-level details of how these entities are implemented in the software. This allows the user to specify the system in terms of Ispecs, and then for the application software to be generated from this description.
An “Event” a particular type of Ispec that is intended for use in keeping track of a chronology of events, such as a history of transactions.
Due to normal business dynamics, users of business applications may frequently want to change the contents of their abstract entities. For example, an ISPEC (entity type) called CUSTOMER might have been initially declared as containing just one address. After the application containing this entity has been in use for some time, business needs may dictate an addition to the entity CUSTOMER of more addresses and phone numbers. It is also possible that the type and role of some data items may change.
For example, a banking application which is migrated from one country to another (say from the UK to Turkey) may require larger decimal numbers to express prices. Frequently, more complicated changes are required—new abstract entities may be added to the application software (like SPECIALCUSTOMER) and some of the new entities may reference existing entities.
In addition, a rethink of an organisation's business rules or a business reorganization may result in the requirement to split and reorganize existing entities into different structures. A change to an existing and working application is easily incorporated by applications such as EAE. The change merely requires the user to change the description of the abstract entity, and EAE subsequently generates a new database schema and new terminal screen layouts. This is in strong contrast to traditional applications where such changes have to be implemented ‘by hand’, which leads to extremely complex and error-prone modifications.
However, the mere generation of a new database schema after changes to the business rules does not complete the change process. An old database with old style data and an old style structure already exists. Therefore, a transformation of the data from the old-style database into a new-style database is required.
The modification of the existing database, to satisfy changes required by business dynamics, is a crucial operation affecting a customer's business. This operation, termed “database reorganisation”, potentially changes the format and the dependency structure of database information.
In other words, a database reorganisation is a set of all operations that are performed on the database to convert the database structure and data contents from an old schema to a new schema. This may include changes to column types and sizes, changes in the number of columns in a table, the creation of new tables, the removal of existing tables, the transformation of the content of existing tables, and the creation and/or modification of additional administrative information (indexes, permissions, etc). In practice, the database reorganisation may vary from trivial (a change in one column of one table) to complex (a change in dozens or hundreds of tables).
Any reorganisation must preserve the existing database data, as the end user may have collected years of business records.
Therefore, the first crucial requirement of the database reorganisation process is its logical consistency and data preservation.
The second, operational requirement of database reorganisation is the minimisation of “downtime” (i.e. the time during which the application and the database cannot be accessed or used by end users). While changes are made to the database, end users are prevented from the database (i.e. they are locked out), to avoid the introduction of inconsistencies into the data. In many production systems, downtime may be a number of hours, or even a number of days, which is generally deemed to be unacceptable. The long downtime is generally due to the fact that the database is large and contains many linked or associated elements—changing a column in just one table may appear to be a one-line operation, but the change may affect gigabytes of data. The situation is compounded when more significant changes, related to dependencies between abstract entities, are made. Therefore, it is desirable to minimise the amount of time taken to reorganise a database.
There are a number of known standard approaches to the reduction of database reorganisation time. The known approaches rely primarily on logical analysis to reduce the number and complexity of required operations. A number of “rules of thumb” are employed. For example, it is acknowledged that there is no need to modify tables that do not need modification.
In some applications, however, to simplify the programming required, no “rules of thumb” are employed, and the entire contents of the database are simply deleted and then recreated.
Neither approach is entirely satisfactory.